Adaptive borescope inspection

ABSTRACT

A method of adaptive inspection includes receiving data characterizing one or more images of an inspection region of an industrial machine acquired by an inspection system operating based on a first set of operating parameters. The inspection region includes a site feature. The method also includes determining, by an analytical model, one or more characteristics of the inspection region from the received data characterizing the one or more images of the inspection region. The method further includes generating a control signal based on the one or more characteristics of the inspection region and/or a user input. The inspection system is configured to perform a new inspection of the inspection region based on the control signal.

BACKGROUND

Video inspection devices, such as video endoscopes or borescopes, can beused to take depth measurements on an object (e.g., lowest points inanomalies such as pits or dents, heights of welds, measurements ofoffsets or clearances between surfaces, etc.). Additionally, videoinspection devices can be used to observe defects (e.g., tears, cracks,scratches, etc.) on a surface of an object (e.g., an industrialmachine). In many instances, the surface of the object is inaccessibleand cannot be viewed without the use of the video inspection device. Forexample, a video inspection device can be used to inspect the surface ofa blade of a turbine engine on an aircraft or power generation unit toidentify any anomalies to determine if any repair or further maintenanceis required. In order to make that assessment, it is often necessary toobtain highly accurate-dimensional measurements of the surface to verifythat the anomaly does not fall outside an operational limit or requiredspecification for that object.

SUMMARY

Various aspects of the disclosed subject matter may provide one or moreof the following capabilities.

In one implementation, a method includes receiving data characterizingone or more images of an inspection region of an industrial machineacquired by an inspection system operating based on a first set ofoperating parameters. The inspection region includes a site feature. Themethod also includes determining, by an analytical model, one or morecharacteristics of the inspection region from the received datacharacterizing the one or more images of the inspection region. Themethod further includes generating a control signal based on the one ormore characteristics of the inspection region and/or a user input. Theinspection system is configured to perform a new inspection of theinspection region based on the control signal.

One or more of the following features can be included in any feasiblecombination.

In one implementation, determining one or more characteristic includesone or more of determining an identity of the site feature by an imagerecognition algorithm in the analytical model. The image recognitionalgorithm receives data characterizing the one or more images of theinspection region and historical images of inspection region as inputs;determining image quality characteristics of the image of the inspectionregion. The image quality characteristics include one or more ofresolution of the image, illumination of the site feature in the image,size of an image of the site feature relative to the size of the imageof the inspection region; and determining site feature characteristicsincluding one or more of size, shape, and depth of the site feature. Inanother implementation, the site feature is a site defect in theinspection region of the industrial machine. Identifying the sitefeature includes determining a defect type associated with the sitedefect.

In one implementation, the method further includes determining that thenew inspection of the inspection region includes acquiring a new image.The determining is based on the identity of the site feature and/orimage quality characteristics of the image of the inspection region. Themethod also includes determining a new set of operating parametersassociated with the inspection system for the acquisition of the newimage. The control signal is based on the new set of operatingparameters.

In one implementation, the determination of acquiring a new image isbased on comparing the image quality characteristics with predeterminedimage quality characteristics comprising one or more of a predeterminedimage resolution, a predetermined image illumination, a predeterminedratio of size of the image of the site feature relative to the size ofthe image of the inspection region. In another implementation, thecontrol signal is configured to vary at least one of a position of alighting device, an orientation of the lighting device and an intensityassociated with the lighting device in the inspection system to a newposition, a new orientation, and a new intensity, respectively. The newset operating parameters include at least one of the new position, thenew orientation and the new intensity.

In one implementation, the control signal is configured to vary at leastone of a position of a lighting device, an orientation of the lightingdevice and an intensity associated with the lighting device in theinspection system to a new position, a new orientation, and a newintensity, respectively. The new set operating parameters include atleast one of the new position, the new orientation and the newintensity.

In one implementation, the new image is a high-fidelity image, and thedetermination of acquiring a new image is based on comparing theidentity of the site feature with a predetermined list of site featuresrequiring high-resolution images. In another implementation, thehigh-fidelity image can include one or more of a high-resolution image,a stereoscopic image, a panoramic image, a high-dynamic range image, a3D point cloud image, a flash mode image, and a live image. In yetanother implementation, the control signal is configured to activate ahigh-resolution camera in the inspection system to capture the newimage. In another implementation, the control signal is configured toactivate one or more cameras in the inspection system to capturemultiple images. The captured multiple images are used to generate oneor more of the panoramic image, the stereoscopic image and the 3D pointcloud image.

In one implementation, the method further includes determining that thenew inspection of the inspection region includes acquiring a video ofthe inspection region, the determining based on comparing the identityof the site feature with a predetermined list of site features requiringvideos. In another implementation, the control signal is configured toactivate a video camera in the inspection system to capture a video.

In one implementation, the method includes determining that the newinspection of the inspection region includes acquiring multiple imagesof the inspection region during a predetermined time period; anddetermining a new set of operating parameters associated with a turningtool of the inspection system. The turning tool is coupled to theinspection region and is configured to move the inspection region. Inanother implementation, the control signal is configured vary a state ofthe turning tool. The control signal is based on the new set ofoperating parameters. In yet another implementation, the control signalis configured to vary a speed of rotation associated with the turningtool and the inspection region relative to the camera. In anotherimplementation, the control signal is configured to move the turningtool back and forth resulting in a back and forth motion of theinspection region relative to the camera. In another implementation, thecontrol signal is configured to move the turning tool such that theresulting motion of the inspection region matches a motion of thecamera.

In one implementation, the method further includes generating a datafile comprising one or more of data characterizing the one or moreimages of an inspection region, the one or more characteristics of theinspection region, the identity of the site feature, image qualitycharacteristics, site feature characteristics, and the new set ofoperating parameters.

In one implementation, the method further includes generating anotification including one or more of data characterizing the one ormore images of an inspection region, the one or more characteristics ofthe inspection region, the identity of the site feature, image qualitycharacteristics, site feature characteristics, and the new set ofoperating parameters. In another implementation, the method furtherincludes presenting the notification to a user; receiving an input fromthe user in response to the presented notification; and generating thecontrol signal based on the received input. The new inspection of theinspection region by the inspection system is based on the controlsignal.

In one implementation, the control signal can be configured to vary atleast one of a position and an orientation of a camera in the inspectionsystem to a new position and a new orientation, respectively. The newset operating parameters include values for at least one of the newposition and the new orientation. In another implementation, the controlsignal is configured to vary at least one of a position of a lightingdevice, an orientation of a lighting device and an intensity associatedwith the lighting device in the inspection system to a new position, anew orientation and a new intensity, respectively. The new set operatingparameters include at least one of the new position and the neworientation. In yet another implementation, the control signal isconfigured to activate a high-resolution camera in the inspection systemto capture a new image.

In one implementation, the method further includes determining that thenew inspection of the inspection region includes acquiring a video ofthe inspection region, the determining based on comparing the identityof the site feature with a predetermined list of site features requiringvideos. The notification includes a recommendation for acquiring thevideo of the inspection region. In another implementation, the inputfrom the user includes data characterizing acceptance of therecommendation for acquiring the view. The control signal is configuredto activate a video camera in the inspection system to capture thevideo.

In one implementation, the method includes determining that the newinspection of the inspection region includes acquiring multiple imagesof the inspection region during a predetermined time period. The methodalso includes determining a new set of operating parameters associatedwith a turning tool of the inspection system. The turning tool iscoupled to the inspection region and is configured to move theinspection region, and the notification includes a recommendation foracquiring the multiple images of the inspection region and the new setof operating parameters associated with the turning tool. In anotherimplementation, the input from the user includes data characterizingacceptance of the recommendation for acquiring the multiple images.

In one implementation, the control signal is configured vary a state ofthe turning tool. The control signal is based on the new set ofoperating parameters. In another implementation, the control signal isconfigured to vary a speed of rotation associated with the turning tooland the inspection region relative to the camera. In yet anotherimplementation, the control signal is configured to move the turningtool back and forth resulting in a back and forth motion of theinspection region relative to the camera. In one implementation, thecontrol signal is configured to move the turning tool such that theresulting motion of the inspection region matches a motion of thecamera. In another implementation, the data characterizing the one ormore images is associated with a video of the inspection region.

Non-transitory computer program products (i.e., physically embodiedcomputer program products) are also described that store instructions,which when executed by one or more data processors of one or morecomputing systems, causes at least one data processor to performoperations herein. Similarly, computer systems are also described thatmay include one or more data processors and memory coupled to the one ormore data processors. The memory may temporarily or permanently storeinstructions that cause at least one processor to perform one or more ofthe operations described herein. In addition, methods can be implementedby one or more data processors either within a single computing systemor distributed among two or more computing systems. Such computingsystems can be connected and can exchange data and/or commands or otherinstructions or the like via one or more connections, including aconnection over a network (e.g. the Internet, a wireless wide areanetwork, a local area network, a wide area network, a wired network, orthe like), via a direct connection between one or more of the multiplecomputing systems, etc.

These and other capabilities of the disclosed subject matter will bemore fully understood after a review of the following figures, detaileddescription, and claims.

BRIEF DESCRIPTION OF THE FIGURES

These and other features will be more readily understood from thefollowing detailed description taken in conjunction with theaccompanying drawings, in which:

FIG. 1 illustrates a flowchart of an exemplary method for inspection ofan inspection site;

FIG. 2 illustrates an exemplary inspection system configured to inspectthe inspection site; and

FIG. 3 illustrates a diagram illustrating an exemplary embodiment of anondestructive testing (NDT) device.

DETAILED DESCRIPTION

Inspection systems (e.g., systems including borescope and/or turningtools) are commonly employed to inspect industrial machine (e.g., powergeneration equipment, oil and gas equipment, aircraft equipment,manufacturing equipment, and the like). The inspection data generatedfrom the inspection can be presented to a human operator for analysis.In some cases, the data analysis can occur after the inspection has beencompleted. As a result, if the inspection data is found to be erroneousor insufficient (e.g., when a defect in the industrial machine cannot beaccurately identified), additional inspection may need to be performed(e.g., the previous inspection may need to be repeated). This processcan be cumbersome and inefficient. Moreover, the time delay between thetwo inspections can be undesirable (e.g., when a defect that threatensthe integrity of the industrial machine is not detected during the firstinspection). Furthermore, analysis by the human operator (e.g., based onhis/her experience) can be slow, error prone and inconsistent. Thisapplication describes systems and methods for analyzing the inspectiondata (e.g., in real-time), and adapting the inspection based on theanalysis. For example, the inspection data can be analyzed and a newinspection step can be determined and/or executed based on the analysis.Additionally or alternately, a human operator can be guided during theinspection process (e.g., by presentation of the inspection data and/orthe data analysis). For example, a notification can be generated thatcan alert the human operator (e.g., in real time) if a need foradditional inspection arises

FIG. 1 illustrates a flowchart of an exemplary method for inspecting aninspection site (e.g., an industrial machine). At step 102, datacharacterizing an image of an inspection region of an industrial machinecan be received (e.g., by a controller of the inspection system). Thereceived data can be acquired by an inspection system operating based ona set of operating parameters (e.g., predetermined operating parameters)and configured to inspect the inspection region of the industrialmachine. The inspection system can include, for example, an inspectiondevice (e.g., a borescope), a turning tool configured to move theinspection region relative to the inspection device, a controller, etc.The inspection region can include one or more site features (e.g., oneor more defects in the industrial machine, a predetermined targetfeature, etc.). The operating parameters can include, for example,positions and articulation angles of camera(s) in the inspection device,arrangement of lighting device(s) during the capture of image(s) by thecamera(s), the type of data to be captured (e.g., 2D image, 3D image,raw or compressed video, geometrically accurate image for dimensioningof features), etc. The inspection device can operate on the set ofoperating parameters during the acquisition of the image of theinspection region.

FIG. 2 illustrates an exemplary inspection site 200 that can beinspected by an inspection system including an inspection device 202(e.g., a borescope) and/or a turning tool 208. The inspection device 202can be communicatively coupled to a controller 204 that can control theoperation of the inspection device 202 and/or the turning tool 208. Thecontroller 204 can generate control signals that drive the operation ofthe inspection device 202 and/or turning tool 208. For example, based onthe control signal, the inspection device 202 can inspect a site feature212 in an inspection region of the inspection site 200. Additionally oralternately, the control signal can instruct the turning tool 208 tomove the inspection site 200 or a portion thereof during or prior to theinspection. For example, the inspection site 200 or a portion thereofcan be immovably coupled to the turning tool 208. As the turning tool208 moves (e.g., rotational motion, translational motion or acombination thereof), the inspection site 200 or a portion thereof canmove.

In some implementations, the operations of the inspection device 202 andthe turning tool 208 can be coordinated. For example, a first controlsignal can instruct the turning tool 208 to place the inspection site200 (or a portion thereof) in a predetermined position and/ororientation (e.g., relative to the inspection device 202); and a secondcontrol signal can instruct the inspection device to perform aninspection (e.g., acquire an image and/or video) when the inspectionsite (or a portion thereof) is in the predetermined position and/ororientation. This process can be repeated resulting in inspection of thedevice in multiple positions/orientation.

The controller 204 can also receive inspection data from the inspectiondevice 202. The received data can include characteristic of propertiesof the inspection site 200 (e.g. temperature, humidity, etc.), acquiredimages, acquired videos, etc. The inspection data can be presented on adisplay screen on the controller 204, or can be transmitted to thecomputing device 206 (where it can be displayed and/or analyzed). Insome implementations, the controller 204 can include a display screenwhere the inspection data, operating parameters of the inspectionsystem, etc., can be displayed.

Returning to FIG. 1, at step 104, one or more characteristics of theinspection region can be determined from the received datacharacterizing the image of the inspection region received at step 102(e.g., by an analytical model). In some implementations, an identity ofthe site feature in the inspection region can be determined. An imagerecognition algorithm (e.g., in the analytical model) can identify thesite feature from the image of the inspection region. The imagerecognition algorithm can be trained on training data including imagesof multiple site features in multiple inspection regions (e.g.,different types of defects in multiple industrial machines). Forexample, the training data can include images of defects (e.g. crack,tear, rub, dent, coating loss, missing material, erosion, excessmaterial, a fissure, or a combination thereof) in industrial machines(e.g., turbines, automotive engines, heat exchangers, industrial piping,etc.). In some implementations, a previously trained image recognitionalgorithm can be stored in a memory device accessible to the controller(e.g., controller 204). In other implementations, the image recognitionalgorithm can be continuously trained during the inspection process(e.g., by using the inspection data as training data). The imagerecognition algorithm can identify the site feature (e.g., “defecttype”). For example, the defect can be identified as one of a crack,tear, rub, dent, coating loss, missing material, erosion, excessmaterial, a fissure, or a combination thereof. In some implementations,the determined characteristic can include image acquisition parametersassociated with the inspection of the inspection region (e.g., lighting,sharpness, brightness, dirty lenses, overly-oblique angles,vibration/motion, etc.).

In some implementations, image quality characteristics of the image ofthe inspection region can be determined. The image qualitycharacteristics can include a resolution of the image. The image qualitycharacteristics can depend on the type of inspection device used toacquire the image, location/orientation of the inspection device,location/orientation of lighting instrument used to illuminate theinspection region during the acquisition of the image, etc. For example,the analytical model can determine if the acquired image has desirableresolution. This can be done, for example, by determining the number ofpixels per unit area of the image or a portion thereof (e.g., portionthat includes the image of the site feature). Additionally oralternately, resolution of the image can be determined based on knowncharacteristics of the camera in the inspection device (e.g., on theresolution of the camera).

The image quality characteristic can include properties of the image ofthe site feature. For example, the analytical model can determine thesize of the image of the site feature relative to the size of the imageof the inspection region (e.g., a ratio between them). This can be done,for example, by identifying the site feature in the image of theinspection region (by the image recognition algorithm). For example, theimage recognition algorithm can identify the contours of the sitefeature (e.g., indicative of shape of the site feature) in the image ofthe inspection region, and calculate a size metric representative of thesize of the site feature in the image (e.g., a size metric based on thelength/width of the size feature). The size metric of the site featurecan be compared to the size of the image of the inspection region (e.g.,by calculating a ratio between the two). The analytical model maydetermine a depth of the site feature (e.g., depth of a crack). Theanalytical model may generate a modified image of the inspection regionwhere the site features are identified (e.g., by superposing a marker onthe site feature). Additionally, a text box including the site featuretype (e.g., defect type) can be placed adjacent to the image of the sitefeature. The image quality characteristic can include brightness of theinspection region image (e.g., based on illumination of the inspectionregion). The illumination of the inspection region can be based onlocation/orientation of lighting instrument used to illuminate theinspection region during the acquisition of the image. In someimplementations, the determined characteristic can include thesurrounding (or environment) of the site feature. For example, the sitefeature can be located on/adjacent to a machine part (e.g., a turbine).The determined characteristic can be the identity of the machine part(s)that can be determined by the analytical model.

At step 106, a control signal can be generated (e.g., by the controllerof the inspection system). The control signal can be generated based onone or more characteristics determined at step 104 and/or based on auser input (e.g., provided via computing device 206, provided via adisplay screen in the controller 204). The control signal that caninstruct the industrial machine to perform a new inspection of theinspection region. The control signal can include the operatingparameters based on which the new inspection needs to be performed. Thenew inspection can include, for example, acquiring a new image (ormultiple new images) of the inspection region (e.g., a high-resolutionimage, a zoomed-in image, a zoomed-out image, an image from a differentperspective, an image having a different illumination, etc.).Additionally or alternately, the new inspection can include acquiring avideo of the inspection region. In some implementations, the newinspection can include stopping the inspection process (e.g., stoppingthe turning tool, switching off the inspection device, or a combinationthereof).

In some implementations, the determination that a new inspection of theinspection region needs to be performed can be based on comparison ofthe characteristics of the inspection region determined at step 104 withpredetermined inspection constraints. The predetermined inspectionconstraints can include predetermined defect type, predetermined imagequality characteristics (e.g., predetermined image resolution, apredetermined image illumination, a predetermined ratio of size of theimage of the site feature relative to the size of the image of theinspection region, etc.). The new inspection of the inspection regioncan be based on correlation between the characteristics determined atstep 104 and the predetermined constraints. The correlation can beindicative of how similar the characteristics are to the predeterminedconstraints. If the corresponding characteristics obtained from theinspection measurement data are within a predetermined range from thepredetermined constraints, the characteristic of the receivedmeasurement data can be considered to correlate with the predeterminedconstraints.

The predetermined constraints can include constraints associated withthe image of the inspection region/site feature in the inspection region(e.g., image resolution, image brightness, size of the site feature inthe image, etc.). In some implementations, the predetermined constraintscan indicate a type of defect. The analytical model can determine thatthe site feature is a defect and can assign a characteristic identifierthat can indicate the defect type. If the characteristic identifier issimilar to a constraint characteristic identifier in the predeterminedconstraints, the received measurement data can be considered tocorrelate with the predetermined constraints.

The control signal can instruct the inspection system (e.g., inspectiondevice, turning tool, or a combination thereof) to execute the newinspection. The control signal can include a new set of operatingparameters for the inspection system. Upon receipt of the controlsignal, the inspection system can perform the new inspection byoperating based on the new set of operating parameters. The controlsignal can be generated, for the controller (e.g., controller 204) andcan be transmitted to the inspection system. In some implementations,the inspection device of the inspection system can include a probedriver that can move one or more portions of the inspection device basedon the control signal (e.g., as described in FIG. 3). One or more motorsin the inspection device can vary the position/orientation of theinspection device (or a portion thereof) to allow for inspection of thesite feature. For example, a head section of the inspection device canbe positioned adjacent to the site feature (e.g., bytranslation/rotation of the head section). The head section can includeone or more sensors (e.g., IR camera, visible-light camera, vibrationdetectors, temperature sensors, etc.), and light sources that can allowfor inspection of the site feature. Details of an exemplary inspectiondevice (a borescope) are provided below with the description of FIG. 3.

In some implementations, the new inspection of the inspection region caninclude acquiring a new image. For example, if the image qualitycharacteristics do not correlate with predetermined image qualityconstraints (e.g., predetermined image resolution, predetermined imagebrightness, predetermined size of image of the site feature ininspection region image, etc.), it can be determined that a new image ofthe inspection region should be acquired by the inspection device.Additionally or alternately, the identity of the site feature (e.g.,defect type) can correlate with a predetermined site feature type thatrequires a new image (e.g., a site feature that requires a high-fidelityimage). The controller can determine the new set of operating parametersassociated with the acquisition of the new image. The operatingparameters can include position/orientation of the camera(s) and/orlighting instruments in the inspection device (e.g., when the acquiredimage of the site feature was not captured from a desired perspective,not captured with a desired illumination, etc.). The new image can beacquired by moving/orienting the camera(s) and/or lighting instrument(s)based on the aforementioned operating parameters (e.g., communicated tothe inspection device via the control signal).

In some implementations, the new set of operating parameters can includefocus values for the camera (e.g., when the ratio of the size of theimage of the site feature and the size of the image of the inspectionregion does not have a desired value, when the acquired image isblurred, etc.). The new image can be acquired by zooming in the camera(e.g., when the ratio is below a low threshold value) or by zooming outthe camera (e.g., when the ratio is above a high threshold value or whenthe image of the entire site feature is not captured). Additionally oralternately, the new image can be acquired by moving the camera towardsthe site feature or moving the camera away from the site feature.

In some implementations, it can be determined that the new inspectioncan include capturing one or more high-fidelity image. In oneimplementation, high-fidelity image can include a high-resolution imageof the inspection region (e.g., when the identified site featurecorrelates with one of the site features in a predetermined list of sitefeatures requiring high-resolution images). The control signal canactivate a high-resolution camera in the inspection device and instructit to acquire a high-resolution image. In some implementations, thehigh-fidelity image can include a panoramic image (e.g., generated bycapturing multiple images of the inspection region and combining theimages to for a panoramic image). The control signal can instruct onemore cameras in the inspection device to capture multiple images andtransmit data characterizing the images to the controller. Thecontroller can combine the images to generate the panoramic image. Insome implementations, the controller can combine the images to form a 3Dpoint cloud image. In some implementations, the high-fidelity image caninclude one or more of a stereoscopic image, a high-dynamic range imageand a live photo.

In some implementations, it can be determined that the new inspectioncan include recording a video of the site feature (e.g., when theidentified site feature correlates with one of the site features in apredetermined list of site features requiring a video). The controlsignal can activate a video camera in the inspection device. The videocamera can be activate for a predetermined period(s) of time (e.g., toperiodically capture the video of a moving machine part).

In some implementations, it can be determined that the new inspectioncan include acquiring multiple images of the inspection region (e.g.,during a predetermined time period). For example, the various images canbe captured by moving the inspection region relative to the inspectiondevice, and capturing the video for various locations/orientations ofthe inspection region (e.g., relative to the inspection device). In someimplementations, the new set of operating parameters can be associatedwith a state of the turning tool (e.g., turning tool 208) and/or motionof the inspection device (e.g., inspection device 202). The state of theturning tool can be indicative of the motion of the turning tool. Forexample, the turning tool can be stationary, translating/rotating with aconstant velocity/angular velocity, rotating back and forth, translatingback and forth, etc. If the turning tool is immovably coupled to theinspection region or a portion thereof (e.g., to a turbine blade of anengine), the inspection region can follow the motion of the turningtool. In one implementation, based on the control signal to the turningtool and/or the inspection device, the motion of the inspection devicecan match the motion of the inspection region. This can allow foracquisition of multiple images of a moving inspection region (e.g., aturbine blade) from a fixed perspective (e.g., during a predeterminedtime period). In some implementations, the control signal can vary theangular/translational velocity of the turning tool/inspection regionrelative to the inspection device.

In some implementations, the control signal can be generated based on auser input (e.g., provided via the computing device 206, via a displayscreen in the controller 204, etc.). For example, a notificationincluding one or more of data characterizing an image of an inspectionregion, the one or more characteristics of the inspection region, theidentity of the site feature, image quality characteristics, sitefeature characteristics, and recommendation for the new inspection(e.g., based on the new set of operating parameters), etc., can begenerated. The notification can be presented to a user who can reviewthe notification and provide an input. The input from the user caninclude an approval for the execution of the new inspection (e.g., basedon the new set of operating parameters). Alternately, the user can makechanges to the new inspection (e.g., by changing one or more of the newset of operating parameters). The user input can be received (e.g., bythe controller 204), and the control signal can be generated based onthe user input (e.g., based on the new set of operating parameter, basedon revised operating parameters provided by the user, etc.). Based onthe user input, a control signal can be generated. The control signalcan include the new set of operating parameters and/or modifications tothe new set of operating parameters (e.g., based on user input). Theinspection system can perform a new inspection based on the controlsignal as described above (e.g., vary at least one of a position and anorientation of a camera/lighting device in the inspection system to anew position/a new orientation; activate a high-resolution camera toacquire a high-resolution image, activate one or more cameras to capturemultiple images to generate a panoramic image/3D point cloud image,activate a video camera, acquire multiple images by varying relativemotion between the inspection region and the inspection device, etc.).

In some implementations, the notification can include an inspectionsummary that can include data characterizing the image received at step102, the characteristics of the inspection region determined at step104, the new inspection including the new operating parameters, etc. Thenotification can include modified images of the inspection region thatinclude image analysis information (e.g., markers around site features,inspection data, etc.) The notification can be provided to anoperator/user. This can be done, for example, via a GUI display space ina computing device (e.g., a laptop, a tablet, computing device 206,controller 204, etc.). The notification can provide an option to theuser to save one or more of the inspection data associated with the sitefeature, operating parameters/constraints of the inspection deviceassociated with the inspection of the site feature, and additionalnotes. Upon receiving instructions from the user, a data file includingthe aforementioned data can be generated and saved (e.g., in a databasein controller 204, computing device 206, cloud database, etc.). The datafile can include one or more of data characterizing the one or moreimages of the inspection region, the one or more characteristics of theinspection region, the identity of the site feature, image qualitycharacteristics, site feature characteristics, and the new set ofoperating parameters. The generated data file can be linked to the sitefeature (e.g., as an icon with an indication of the site feature), andcan be provided to a user during the inspection. For example, the datafile can be presented as an icon in the GUI display space. The user canaccess the data file by clicking on the icon.

In some implementations, the notification can include the recommendationfor acquiring the video of the inspection region (e.g., which can bedetermined based on comparison between the identity of the site featurewith a predetermined list of site features requiring videos). In someimplementations, the notification can include a recommendation foracquiring multiple images of the inspection region and the new set ofoperating parameters associated with the turning tool. As describedabove, the control signal can be generated based on the input providedby the user in response to the recommendation in the notification.

FIG. 3 is a diagram illustrating an exemplary embodiment of aninspection device (e.g., a non-destructive device) in the form of aborescope 300. The borescope 300 can include a control unit 302 (orcontroller 204), a conduit section 304, a bendable articulation section306, and a head section 308. In one embodiment, the sections 304, 306,308 can have different lengths and can be integral with one another, orcan be detachable from one another. As depicted, the conduit section 304is suitable for insertion into a variety of different targets, such asinside turbomachinery, equipment, pipes, conduits, underwater locations,curves, bends, inside or outside of an aircraft system, and the like.

The borescope 300 can include a probe driver 309 coupled to the conduitsection 304. The probe driver 309 can include a motor (not shown)configured to translate and/or rotate one or more of the sections 304,306, 308 (e.g., to facilitate insertion of the probe head 308 into thetarget). Additionally or alternatively, orientation/position of aportion of the head section 308 (e.g., camera, light source, etc.) canbe varied to acquire an inspection region image (e.g., RGB image, IRimage, etc.). The control unit 302 can include a control unit housing310, a controller 312, a directional input 314, and a screen 316. Thecontroller 312 can include a processor 318 and a readable memory 320containing computer readable instructions which can be executed by theprocessor 318 in order to actuate the borescope 300. The computerreadable instructions can include an inspection plan based on which theborescope 300 or a portion thereof (e.g., a conduit section 304, abendable articulation section 306, and a head section 308) can betranslated/rotated (e.g., by the probe driver 309). In someimplementations, the operation of the probe driver 309 can be based on acontrol signal (e.g., generated by the controller 204 based on theinspection plan/user input via GUI display space on screen 316 or acomputing device, etc.).

The controller 312 can be communicatively coupled to the control unit302 via one or more signals 321. The controller 312 can also be arrangedwithin the control unit housing 310, or can be arranged outside thecontrol unit housing 310. On some implementations, the directional input314 can be configured to receive user input (e.g., direction controls)to the control unit 302 for actuation of the borescope 300. The screen316 can display visual information being received by the camera(comprising an optical sensor) arranged in the head section 308, whichcan allow the user to better guide the borescope 300 using thedirectional input 314. The directional input 314 and the screen 316 canbe communicatively coupled to the controller 312 via the one or moresignals 321, which can be a hard-wired connection or a wireless signal,such as WI-FI or Bluetooth. In one implementation, inspection dataand/or notifications (e.g., notifications based on inspection data asdescribed above) can be provided on the screen 316.

The conduit section 304 can include a tubular housing 322 including aproximal end 324 and a distal end 326. The tubular housing 322 can be aflexible member along its whole length, or can be rigid at the proximalend 324 and become more flexible travelling down the length of theconduit section 304 towards the distal end 326. In certain embodiments,the tubular housing 322 can be formed from a non-porous material toprevent contaminants from entering the borescope 300 via the conduitsection 304.

The control unit 302 can be arranged at the proximal end 324 of thetubular housing 322, and the bendable articulation section 306 can bearranged at the distal end of the tubular housing 322. The bendablearticulation section 306 can include a bendable neck 328 and washers330. The bendable neck 328 can be arranged at the distal end 326 of thetubular housing 322, and is able to be actuated 360° in the Y-Z plane.The bendable neck 328 can be wrapped in a non-porous material to preventcontaminants from entering the borescope 300 via the bendablearticulation section 306.

The head section 308 can include a head assembly 332. The head assembly332 can include one or more light source 334 (e.g., LEDs or a fiberoptic bundle with lights at the proximal end), a camera 336 (or multiplecameras such as visible-light camera, IR camera, etc.), and one or moresensors 338 that can be configured to collect data about the surroundingenvironment. The camera 336 of the borescope 300 can provide images andvideo suitable for inspection to the screen 316 of the control unit 302.The light source 334 can be used to provide for illumination when thehead section 308 is disposed in locations having low light or no light.The sensor 338 can record data including temperature data, distancedata, clearance data (e.g., distance between a rotating element and astationary element), flow data, and so on.

In certain embodiments, the borescope 300 includes a plurality ofreplacement head assemblies 332. The head assemblies 332 can includetips having differing optical characteristics, such as focal length,stereoscopic views, 3-dimensional (3D) phase views, shadow views, etc.Additionally or alternatively, the head section 308 can include aremovable and replaceable portion of the head section 308. Accordingly,a plurality of the head sections 308, bendable necks 328, and conduitsection 304 can be provided at a variety of diameters from approximatelyone millimeter to ten millimeters or more.

During use, the bendable articulation section 306 and the probe driver309 can be controlled, for example, by the control inputs (e.g.,relative control gestures, physical manipulation device) from thedirectional input 314 and/or control signals generated by the controller312. The directional input can be a joystick, D-pad, touch pad,trackball, optical sensor, or a touchscreen over the screen 316. Thedirectional input 314 can also be a similar device that is locatedoutside the control unit housing 310 and connected by wire or wirelessmeans. In particular, a set of control inputs can be used to control thebendable articulation section 306 and/or the probe driver 309. Thebendable articulation section 306 can steer or “bend” in variousdimensions, while the conduit section 304 can translate and/or rotate,using any combination of actuators and wires arranged within the controlunit 302, to adjust the orientation (e.g., a positioning) of the headsection 308. In some implementations, the control inputs/direction input314 can be generated by the controller based on the inspection plan.

The actuators can be electric, pneumatic, or ultrasonically operatedmotors or solenoids, shape alloy, electroactive polymers, dielectricelastomers, polymer muscle material, or other materials. For example,the bendable articulation section 306 and the probe driver 309 canenable movement of the head section 308 in an X-Y plane, X-Z plane,and/or Y-Z plane. Indeed, the directional input 314 can be used toperform control actions suitable for disposing the head section 308 at avariety of angles, such as the depicted angle α. In this manner, thehead section 308 can be positioned to visually inspect desiredlocations.

Once the head section 308 is in a desired position, the camera 336 canoperate to acquire, for example, a stand-still visual image or acontinuous visual image, which can be displayed on the screen 316 of thecontrol unit 302, and can be recorded by the borescope 300. Inembodiments, the screen 316 can be multi-touch touch screens usingcapacitance techniques, resistive techniques, infrared grid techniques,and the like, to detect the touch of a stylus and/or one or more humanfingers. Additionally or alternatively, acquired visual images can betransmitted into a separate storage device for later reference.

Certain exemplary embodiments will now be described to provide anoverall understanding of the principles of the structure, function,manufacture, and use of the systems, devices, and methods disclosedherein. One or more examples of these embodiments are illustrated in theaccompanying drawings. Those skilled in the art will understand that thesystems, devices, and methods specifically described herein andillustrated in the accompanying drawings are non-limiting exemplaryembodiments and that the scope of the present invention is definedsolely by the claims. The features illustrated or described inconnection with one exemplary embodiment may be combined with thefeatures of other embodiments. Such modifications and variations areintended to be included within the scope of the present invention.Further, in the present disclosure, like-named components of theembodiments generally have similar features, and thus within aparticular embodiment each feature of each like-named component is notnecessarily fully elaborated upon.

The subject matter described herein can be implemented in digitalelectronic circuitry, or in computer software, firmware, or hardware,including the structural means disclosed in this specification andstructural equivalents thereof, or in combinations of them. The subjectmatter described herein can be implemented as one or more computerprogram products, such as one or more computer programs tangiblyembodied in an information carrier (e.g., in a machine-readable storagedevice), or embodied in a propagated signal, for execution by, or tocontrol the operation of, data processing apparatus (e.g., aprogrammable processor, a computer, or multiple computers). A computerprogram (also known as a program, software, software application, orcode) can be written in any form of programming language, includingcompiled or interpreted languages, and it can be deployed in any form,including as a stand-alone program or as a module, component,subroutine, or other unit suitable for use in a computing environment. Acomputer program does not necessarily correspond to a file. A programcan be stored in a portion of a file that holds other programs or data,in a single file dedicated to the program in question, or in multiplecoordinated files (e.g., files that store one or more modules,sub-programs, or portions of code). A computer program can be deployedto be executed on one computer or on multiple computers at one site ordistributed across multiple sites and interconnected by a communicationnetwork.

The processes and logic flows described in this specification, includingthe method steps of the subject matter described herein, can beperformed by one or more programmable processors executing one or morecomputer programs to perform functions of the subject matter describedherein by operating on input data and generating output. The processesand logic flows can also be performed by, and apparatus of the subjectmatter described herein can be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application-specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processor of any kind of digital computer. Generally, aprocessor will receive instructions and data from a Read-Only Memory ora Random Access Memory or both. The essential elements of a computer area processor for executing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto-optical disks, or optical disks. Information carrierssuitable for embodying computer program instructions and data includeall forms of non-volatile memory, including by way of examplesemiconductor memory devices, (e.g., EPROM, EEPROM, and flash memorydevices); magnetic disks, (e.g., internal hard disks or removabledisks); magneto-optical disks; and optical disks (e.g., CD and DVDdisks). The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

To provide for interaction with a user, the subject matter describedherein can be implemented on a computer having a display device, e.g., aCRT (cathode ray tube) or LCD (liquid crystal display) monitor, fordisplaying information to the user and a keyboard and a pointing device,(e.g., a mouse or a trackball), by which the user can provide input tothe computer. Other kinds of devices can be used to provide forinteraction with a user as well. For example, feedback provided to theuser can be any form of sensory feedback, (e.g., visual feedback,auditory feedback, or tactile feedback), and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The techniques described herein can be implemented using one or moremodules. As used herein, the term “module” refers to computing software,firmware, hardware, and/or various combinations thereof. At a minimum,however, modules are not to be interpreted as software that is notimplemented on hardware, firmware, or recorded on a non-transitoryprocessor readable recordable storage medium (i.e., modules are notsoftware per se). Indeed “module” is to be interpreted to always includeat least some physical, non-transitory hardware such as a part of aprocessor or computer. Two different modules can share the same physicalhardware (e.g., two different modules can use the same processor andnetwork interface). The modules described herein can be combined,integrated, separated, and/or duplicated to support variousapplications. Also, a function described herein as being performed at aparticular module can be performed at one or more other modules and/orby one or more other devices instead of or in addition to the functionperformed at the particular module. Further, the modules can beimplemented across multiple devices and/or other components local orremote to one another. Additionally, the modules can be moved from onedevice and added to another device, and/or can be included in bothdevices.

The subject matter described herein can be implemented in a computingsystem that includes a back-end component (e.g., a data server), amiddleware component (e.g., an application server), or a front-endcomponent (e.g., a client computer having a graphical user interface ora web interface through which a user can interact with an implementationof the subject matter described herein), or any combination of suchback-end, middleware, and front-end components. The components of thesystem can be interconnected by any form or medium of digital datacommunication, e.g., a communication network. Examples of communicationnetworks include a local area network (“LAN”) and a wide area network(“WAN”), e.g., the Internet.

Approximating language, as used herein throughout the specification andclaims, may be applied to modify any quantitative representation thatcould permissibly vary without resulting in a change in the basicfunction to which it is related. Accordingly, a value modified by a termor terms, such as “about” and “substantially,” are not to be limited tothe precise value specified. In at least some instances, theapproximating language may correspond to the precision of an instrumentfor measuring the value. Here and throughout the specification andclaims, range limitations may be combined and/or interchanged, suchranges are identified and include all the sub-ranges contained thereinunless context or language indicates otherwise.

1. A method comprising: receiving data characterizing one or more imagesof an inspection region of an industrial machine acquired by aninspection system operating based on a first set of operatingparameters, wherein the inspection region includes a site feature;determining, by an analytical model, one or more characteristics of theinspection region from the received data characterizing the one or moreimages of the inspection region; and generating a control signal basedon the one or more characteristics of the inspection region, wherein theinspection system is configured to perform a new inspection of theinspection region based on the control signal.
 2. The method of claim 1wherein determining one or more characteristic includes one or more of:determining an identity of the site feature by an image recognitionalgorithm in the analytical model, wherein the image recognitionalgorithm receives data characterizing the one or more images of theinspection region and historical images of inspection region as inputs;determining image quality characteristics of the image of the inspectionregion, wherein the image quality characteristics include one or more ofresolution of the image, illumination of the site feature in the image,size of an image of the site feature relative to the size of the imageof the inspection region; and determining site feature characteristicsincluding one or more of size, shape, and depth of the site feature. 3.The method of claim 2, wherein the site feature is a site defect in theinspection region of the industrial machine, wherein identifying thesite feature includes determining a defect type associated with the sitedefect.
 4. The method of claim 2, further comprising: determining thatthe new inspection of the inspection region includes acquiring a newimage, the determining based on the identity of the site feature and/orimage quality characteristics of the image of the inspection region; anddetermining a new set of operating parameters associated with theinspection system for the acquisition of the new image, wherein thecontrol signal is based on the new set of operating parameters.
 5. Themethod of claim 4, wherein the determination of acquiring a new image isbased on comparing the image quality characteristics with predeterminedimage quality characteristics comprising one or more of a predeterminedimage resolution, a predetermined image illumination, a predeterminedratio of size of the image of the site feature relative to the size ofthe image of the inspection region.
 6. The method of claim 5, whereinthe control signal is configured to vary at least one of a position andan orientation of a camera in the inspection system to a new positionand a new orientation, respectively, wherein the new set operatingparameters include values for at least one of the new position and thenew orientation.
 7. The method of claim 5, wherein the control signal isconfigured to vary at least one of a position of a lighting device, anorientation of the lighting device and an intensity associated with thelighting device in the inspection system to a new position, a neworientation, and a new intensity, respectively, wherein the new setoperating parameters include at least one of the new position, the neworientation and the new intensity.
 8. The method of claim 4, wherein thenew image is a high-fidelity image, and wherein the determination ofacquiring a new image is based on comparing the identity of the sitefeature with a predetermined list of site features requiringhigh-resolution images.
 9. The method of claim 8, wherein thehigh-fidelity image can include one or more of a high-resolution image,a stereoscopic image, a panoramic image, a high-dynamic range image, a3D point cloud image, a flash mode image, and a live image.
 10. Themethod of claim 9, wherein the control signal is configured to activatea high-resolution camera in the inspection system to capture the newimage.
 11. The method of claim 9, wherein the control signal isconfigured to activate one or more cameras in the inspection system tocapture multiple images, wherein the captured multiple images are usedto generate one or more of the panoramic image, the stereoscopic imageand the 3D point cloud image.
 12. The method of claim 2, furthercomprising: determining that the new inspection of the inspection regionincludes acquiring a video of the inspection region, the determiningbased on comparing the identity of the site feature with a predeterminedlist of site features requiring videos.
 13. The method of claim 12,wherein the control signal is configured to activate a video camera inthe inspection system to capture a video.
 14. The method of claim 2,further comprising: determining that the new inspection of theinspection region includes acquiring multiple images of the inspectionregion during a predetermined time period; and determining a new set ofoperating parameters associated with a turning tool of the inspectionsystem, wherein the turning tool is coupled to the inspection region andis configured to move the inspection region.
 15. The method of claim 14,wherein the control signal is configured to vary a state of the turningtool, wherein the control signal is based on the new set of operatingparameters.
 16. The method of claim 15, wherein the control signal isconfigured to vary a speed of rotation associated with the turning tooland the inspection region relative to the camera.
 17. The method ofclaim 15, wherein the control signal is configured to move the turningtool back and forth resulting in a back and forth motion of theinspection region relative to the camera.
 18. The method of claim 15,wherein the control signal is configured to move the turning tool suchthat the resulting motion of the inspection region matches a motion ofthe camera.
 19. The method of claim 4, further comprising generating adata file comprising one or more of data characterizing the one or moreimages of an inspection region, the one or more characteristics of theinspection region, the identity of the site feature, image qualitycharacteristics, site feature characteristics, and the new set ofoperating parameters.
 20. The method of claim 4, further comprising:generating a notification including one or more of data characterizingthe one or more images of an inspection region, the one or morecharacteristics of the inspection region, the identity of the sitefeature, image quality characteristics, site feature characteristics,and the new set of operating parameters.
 21. The method of claim 20,further comprising: presenting the notification to a user; receiving aninput from the user in response to the presented notification;generating the control signal based on the received input, wherein thenew inspection of the inspection region by the inspection system isbased on the control signal.
 22. The method of claim 21, wherein thecontrol signal is configured to vary at least one of a position and anorientation of a camera in the inspection system to a new position and anew orientation, respectively, wherein the new set operating parametersinclude values for at least one of the new position and the neworientation.
 23. The method of claim 21, wherein the control signal isconfigured to vary at least one of a position of a lighting device, anorientation of the lighting device and an intensity associated with thelighting device in the inspection system to a new position, a neworientation, and a new intensity, respectively, wherein the new setoperating parameters include at least one of the new position, the neworientation and the new intensity.
 24. The method of claim 21, whereinthe control signal is configured to activate a high-resolution camera inthe inspection system to capture a new image.
 25. The method of claim21, further comprising determining that the new inspection of theinspection region includes acquiring a video of the inspection region,the determining based on comparing the identity of the site feature witha predetermined list of site features requiring videos, wherein thenotification includes a recommendation for acquiring the video of theinspection region.
 26. The method of claim 25, wherein the input fromthe user includes data characterizing acceptance of the recommendationfor acquiring the view, and wherein the control signal is configured toactivate a video camera in the inspection system to capture the video.27. The method of claim 21, further comprising: determining that the newinspection of the inspection region includes acquiring multiple imagesof the inspection region during a predetermined time period; anddetermining a new set of operating parameters associated with a turningtool of the inspection system, wherein the turning tool is coupled tothe inspection region and is configured to move the inspection region,wherein the notification includes a recommendation for acquiring themultiple images of the inspection region and the new set of operatingparameters associated with the turning tool.
 28. The method of claim 27,wherein the input from the user includes data characterizing acceptanceof the recommendation for acquiring the multiple images.
 29. The methodof claim 28, wherein the control signal is configured vary a state ofthe turning tool, wherein the control signal is based on the new set ofoperating parameters.
 30. The method of claim 28, wherein the controlsignal is configured to vary a speed of rotation associated with theturning tool and the inspection region relative to the camera.
 31. Themethod of claim 28, wherein the control signal is configured to move theturning tool back and forth resulting in a back and forth motion of theinspection region relative to the camera.
 32. The method of claim 28,wherein the control signal is configured to move the turning tool suchthat the resulting motion of the inspection region matches a motion ofthe camera.
 33. The method of claim 1, wherein the data characterizingthe one or more images is associated with a video of the inspectionregion.
 34. A system comprising: at least one data processor; memorycoupled to the at least one data processor, the memory storinginstructions to cause the at least one data processor to performoperations comprising: receiving data characterizing one or more imagesof an inspection region of an industrial machine acquired by aninspection system operating based on a first set of operatingparameters, wherein the inspection region includes a site feature;determining, by an analytical model, one or more characteristics of theinspection region from the received data characterizing the one or moreimages of the inspection region; and generating a control signal basedon the one or more characteristics of the inspection region, wherein theinspection system is configured to perform a new inspection of theinspection region based on the control signal.
 35. A computer programproduct comprising a non-transitory machine-readable medium storinginstructions that, when executed by at least one programmable processorthat comprises at least one physical core and a plurality of logicalcores, cause the at least one programmable processor to performoperations comprising: receiving data characterizing one or more imagesof an inspection region of an industrial machine acquired by aninspection system operating based on a first set of operatingparameters, wherein the inspection region includes a site feature;determining, by an analytical model, one or more characteristics of theinspection region from the received data characterizing the one or moreimages of the inspection region; and generating a control signal basedon the one or more characteristics of the inspection region, wherein theinspection system is configured to perform a new inspection of theinspection region based on the control signal.